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DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

2. Claims 1, 2, and 6 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent 5,682,469 to Linnett et ai (herein referred to as "Linnett et al.") 

3. With regard to claim 1, Linnett et al discloses "a method for presenting and controlling a 
digital background for a real-world setting in a computer display, the method comprising the 
following operations:" 

"presenting a background" (Fig. 2); 

"inputting an operator's choice of action or inaction to a situation state module;"(//«es 
15-18 of column 9: "The entering of a manual carriage return by the user is a triggering 
event. The application 28 then tells the tracking service 105 of the occurrence of the 
triggering event (step 107). "; lines 11-2 of column 11 "The shell 30 provides the user 
with the ability to log on and gain access to the rooms that constitute his "home".) "or the 
operator's inaction" {lines 38-41 of column 10: "The idle animation is displayed 
whenever no other commands have been send to the actor services 112. It should be 
appreciated that the idle animation may be requested by applications 28 and by the shell 
30. "); 
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"updating a current state based on the operator's action or inaction" (lines 20-23 of 
column 9: "Each time that the user performs the correct action (i.e., an action that if 
performed incorrectly would trigger a tip), the number of lives is decremented by 1. "; 
lines 12-15 of column 11: "The shell 30 provides the user with the ability to log on and 
gain access to the rooms that constitute his "home". FIG. 12 is a flow chart of the steps 
that are performed during a log-on sequence in the preferred embodiment of the present 
invention. "); 

"using the current state by the decision logic to determine how to control a background 
in the setting" (the state of user *s "first logon " 128 controlling the showing of the last 
room 132 or default room 130 is taught in Fig. 12); 

4. Linnett et al does not use the explicit language "background"; however, one of ordinary 
skill in the art would recognize from Fig. 2 that the room 32 is analogous to a background, as 
broadly recited in the instant application, on which the animated character is placed. 

5. With regard to claim 2, Linnett et al discloses "sending a selection of control of the 
background to the display" (lines 35-37 of column 12: "A user may change the style of a room. 
The shell 30 provides a balloon that provides a user the option of changing the style of the 
room. "). 

6. With regard to claim 6, Linnett et al discloses "using the current state by a decision logic 
to determine a response in a setting by a character" (Fig. 11 A shows the animation 115 being 
requested from the character data file 39 in response to a command 114 triggered by the 
application 28, and Fig. 10B shows the decision logic 110 corresponding to displaying a tip.); 
"modifying the character" (lines 4-8 of column 6: " ...the appearance of the personal character 
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changes. For example, if the user invokes a spreadsheet program, the personal character might 
don a green visor, change his vocabulary and be more helpful than usual "). 

7. Claims 1, 3 and 5 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent 4,766,541 to Bleich et al (herein referred to as "Bleich et al.") 

8. With regard to claim 1, Bleich et al discloses "a method for presenting and controlling a 
digital background for a real-world setting in a computer display, the method comprising the 
following operations:" 

"presenting a background" {lines 42-43 of column 2: "The video game employing the 
interactive video disk background generation system of the present invention is shown as 
10 in FIG. 1..."); 

"inputting an operator's choice of action or inaction to a situation state module; updating 
a current state based on the operator's action or inaction" {lines 21-25 of column 10: 
"One such example is a road-rally type game wherein the player, i.e. the driver, reaches 
an intersection having three possible courses of action: to go straight, to turn right, or to 
turn left Any of these possible sequences may be selected by the player... "); 
"and using the current state by a decision logic to determine how to control the 
background in the setting" {lines 45-48 of column 9: "Foremost is the present capability 
of generating a detailed full screen background which is immediately responsive to 
player or processor directed game play. "; "Interactive video disk game play is achieved 
by game processor 72 control of disk player 12 through interface 78 and, more 
specifically, by selectable skipping of video tracks on the disk. " ). 
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9. Bleich et al does not explicitly use the language "simulation state module" or "current 
state"; however, one of ordinary skill in the art would recognize that it is inherent that a video 
game program as described by Bleich et al in lines 19-15 of column 10 would have a current 
situational state of the player in the game from the statement on lines 23-25 of column 6 {"These 
'movements of game symbols are accompanied by corresponding manipulations of the symbols 
in screen RAM 86. "), and the update of the current situational state would be a result of the 
operator's choice of action or inaction from lines 14-17 of column 6 ( "A variety of game symbols 
may be written to screen RAM 86 including symbols representative of the players, their 
adversaries, as well as obstacles, projectiles, and like objects. "). 

10. With regard to claim 3, Bleich et al discloses "the operation of interacting with a video 
controller to modify the background to a display" {lines 19-21 of column 10: "By contrast, the 
present controller permits alternative background environment segments to be conveniently 
stored in alternating sequences of ^mini-blocks ; lines 25-30 of column 10: "Any of these 
possible sequences may be selected by the player and, since it cannot be known until the player 
reaches the intersection which route will be selected, each is stored on the video disk in a series 
of mini-blocks instantaneously accessible upon proper game processor command. "). 

1 1 . With regard to claim 5, Bleich et al discloses "the background is a digital background," 
{lines 47-48 of column 2: "Video picture data is retained digitally on disk... ") and "wherein the 
operation of presenting the background comprises providing a series of concatenated still 
pictures generated to provide life-like movement in the background" {lines 30-34 of column 10: 
"One possible arrangement of these sequences would be ten frames of left-turn followed by ten 
frames of straight followed by ten frames of right-turn— this sequence being repeated along the 
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disk for the desired time duration. "). Bleich et al does not explicitly use the language "still 
pictures"; however, one of ordinary skill in the art would recognize this feature is inherent by the 
statement on lines 3-8 of column 8: "As previously discussed, each frame of video defines a 
single complete * track ' on the video disk and, therefore, it is merely necessary to incrementally 
displace the playback head radially across the tracks in order to achieve video still-frame or 
frame skipping, in either the forward or reverse direction. 

12. Claims 1 and 9 are rejected under 35 U.S.C. 102(b) as being anticipated by Japanese 
Patent No. 10-201954 A to Kanbe et al (herein referred to as "Kanbe et al") # 

A. 

13. With regard to claim 1, Kanbe et al discloses 

a. "presenting a background" {drawing 2 shows the playing area which also serves 
as the background)', 

b. inputting a human operator's choice of action or the operator's inaction to a 
situation state module" using the controller 16 {machine translation of paragraph 70: 
"That is, CPU 8 generates the command as a task for drawing or a voice output suitably 
based on the contents of directions (...a discarded tile, reach a tile... etc.) directed from a 
game player through a controller 16. "); 

c. "updating a current state based on the operator's action or inaction," wherein the 
operator's actions, such as discarding a tile, can cause an update to the mah-jongg game 
state as shown in machine translated paragraphs 29 and 94; 

d. "using the current state by the decision logic to determine how to control the 
background in the setting" {drawing 2 shows the playing area showing the current state 
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of the game; machine translation of paragraph 5: "The control section which obtains 
image data and/or voice data so that it may be in the display and/or output state 
according to the waging-war situation of said waging-war character... "). 

14. With regard to claim 9, Kanbe et al shows "the operation of presenting a two- 
dimensional representation of a playing area" (drawing 2 shows a two-dimensional top-down 
representation the playing area). 

Claim Rejections - 35 USC §103 

15. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

16. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1. Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

17. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
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invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

18. Claims 4, 7, 11, 14, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Linnett et al in view of U.S. Patent 6,600,491 to Szeliski et al (herein 
referred to as "Szeliski et al.") 

19. With regard to claim 11, Linnett et al discloses "a method for presenting a character for 
an interaction, the method comprising the following operations:" 

"inputting a human operator's choice of action"(/zfl£? 15-18 of column 9: "The entering 
of a manual carriage return by the user is a triggering event. The application 28 then 
tells the tracking service 105 of the occurrence of the triggering event (step 107). "; lines 
11-12 of column 11 "The shell 30 provides the user with the ability to log on and gain 
access to the rooms that constitute his "home".) "or the operator's inaction" (lines 38-41 
of column 10: "The idle animation is displayed whenever no other commands have been 
send to the actor services 112. It should be appreciated that the idle animation may be 
requested by applications 28 and by the shell 30. "); 

"updating a current state based on the operator's action or inaction" (lines 20-23 of 
column 9: "Each time that the user performs the correct action (i.e., an action that if 
performed incorrectly would trigger a tip), the number of lives is decremented by 1. 
lines 12-15 of column 11: "The shell 30 provides the user with the ability to log on and 
gain access to the rooms that constitute his "home". FIG. 12 is a flow chart of the steps 
that are performed during a log-on sequence in the preferred embodiment of the present 
invention. "); 
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"using the current state by a decision logic to determine a response in a setting by the 
character" (Fig. 11 A shows the animation 115 being requested from the character data 
file 39 in response to a command 114 triggered by the application 28, and Fig. 10B 
shows the decision logic 110 corresponding to displaying a tip.); 
"modifying the character" (lines 4-8 of column 6: "...the appearance of the personal 
character changes. For example, if the user invokes a spreadsheet program, the personal 
character might don a green visor, change his vocabulary and be more helpful than 
usual. "). 

20. With regard to the limitation of claim 11 on lines 17-18 of page 20, and claim 7, Linnett 
et al discloses "presenting a series of animation clips for the character" (lines 15-16 of column 
10: "The animations 40 stored within the data file 39 includes video frames as well as duration 
and timing information. "; lines 18-21 of column 10: "The conditional branching information 
allows the play sequence of frames to vary so that, under certain conditions, the frames are 
played at a different sequence than they are normally played. "; lines 38-39 of column 10: "The 
idle animation is displayed whenever no other commands have been sent to the actor services 
112. Linnett et al is silent as to whether the video clips are joined into the appearance of a 
continuous streaming image of the character. Szeliski et al discloses "presenting a series of 
individual video clips that are joined in to the appearance of a continuous streaming image" 
(lines 4-9 of column 13: "Rather, the video clip could be made up of multiple sequences of the 
scene captured at different times. Regardless of how many video sequences make up the inputted 
video clip, the trick is to produce the aforementioned new sequences such that the motion 
appears smooth and seamless to the viewer. ") of a character (lines 43-45 of column 2: "For 
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example, another application of the video sprite concept involves objects that move about the 
scene in the input video clip such as an animal, vehicle, and person. "). 

21 . Linnett et al and Szeliski et al are analogous art because they are from the same problem 
solving area: simulating realistic interaction with computer animated objects. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to modify the 
character animation disclosed by Linnett et al with the video sprite disclosed by Szeliski et al in 
order to give the character the appearance of a continuous streaming image. The motivation for 
doing so would have been to achieve a greater degree of realism by giving the character 
"dynamic qualities" as stated by Szeliski et al in lines 9-1 1 of column 2. Therefore, it would have 
been obvious to combine Linnett et al with Szeliski et al to obtain the invention as specified in 
claims 7 and 11. 

22. With regard to claim 25, Linnett et al discloses "using the current state by the decision 
logic to determine how to control a background in the setting" {the state of user 9 s "first logon " 
128 controlling the showing of the last room 132 or default room 130 is taught in Fig. 12); 
"sending a selection of control of the background to the display" (lines 35-37 of column 12: "A 
user may change the style of a room. The shell 30 provides a balloon that provides a user the 
option of changing the style of the room. "). Linnett et al does not use the explicit language 
"background"; however, one of ordinary skill in the art would recognize from Fig. 2 that the 
room 32 is analogous to a background, as broadly recited in the instant application, on which the 
animated character is placed. 

23. With regard to claims 4 and 24 and the limitation of claim 25 recited on line 16 of page 
15, Linnett et al does not disclose using a continuously streaming video to present a background. 
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Szeliski et al discloses "the operation of continuously streaming video to present a background" 
(line 66 of column 1 through line 1 of column 2: "The new medium, which is referred to as a 
video texture, can provide a continuous, infinitely varying stream of video images. "; line 65 of 
column 21 through line 1 of column 22 as shown in the rejection of claim 77; lines 16-18 of 
column 2: "Video textures could also find application as dynamic backdrops for scenes 
composited from live and synthetic elements. "). 

24. At the time of the invention it would have been obvious to a person of ordinary skill in 
the art to further modify Linnett et al with video textures as taught by Szeliski et al to use 
continuously streaming video to present the background. The motivation for doing so would 
have been to achieve a greater degree of realism by giving the background "dynamic qualities," 
as suggested by Szeliski et al in lines 9-1 1 of column 2. Therefore, it would have been obvious 
to combine Linnett et al with Szeliski et al to obtain the invention as specified in claims 4, 24 
and 25. 

25. Claim 14 is met by the combination of Linnett et al and Szeliski et al, wherein Linnett et 
al discloses "providing a motion of the character in a time-dependent manner based on 
interaction with the human operator" (lines 38-41 of column 10: "The idle animation is displayed 
whenever no other commands have been send to the actor services 112. It should be appreciated 
that the idle animation may be requested by applications 28 and by the shell 30. "). 

26. Claims 11, 12, 15, 16, 18, 19, 20, 22, 23, 26, 28, 30-34, 80, 87, 89, and 90 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Kanbe et al in view of Szeliski et al. 

27. With regard to claim 11, Kanbe et al discloses "a method for presenting a character for 
an interaction, the method comprising the following operations:" 
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e. "inputting a human operator's choice of action" using the controller 16 (machine 
translation of paragraph 70: "That is, CPU 8 generates the command as a task for 
drawing or a voice output suitably based on the contents of directions (...a discarded tile, 
reach a tile, [other game actions], etc.) directed from a game player through a controller 
16."); 

f. "updating a current state based on the operator's action or inaction," wherein the 
operator's actions, such as discarding a tile, can cause an update to the mah-jongg game 
state as shown in machine translated paragraphs 29 and 94; 

g. "using the current state by a decision logic to determine a response in a setting by 
the character" (machine translation of paragraph 1 7: "Furthermore, a motion of the face 
for every waging-war character as image data in this invention is a motion which looks 
at which direction at least among the direction into which the tile was thrown away, the 
direction of the partner who cried, and the direction of a partner which carried out win 
preferably. "; machine translation of paragraph 75: "As shown in drawing 12 , by 
making the example of the CPU character into Uchida (woman), first, a control section 
20 is controlled by the step ST 11 with game data, and the initial set (128 points) of the 
basic feeling value classified by CPU character is performed. "). The computer- 
controlled players respond to the current state in a number of ways: giving messages 
(machine translated paragraph 75), facial expressions (machine translated paragraph 7), 
and game actions in general such as discarding tiles (machine translated paragraph 5); 

h. "modifying the character," wherein the image depicting the character's expression 
is modified based on the feeling value (machine translation of paragraph 43: 
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"Furthermore, with the expression set of the face by the set feeling value, the expression 
of the face of the CPU character is usually changed to four kinds of a laughter face, 
common face, and sadness face and a resentment face according to the set feeling 
value. "). 

28. Kanbe et al does not specifically disclose a video clips joined in to the appearance of a 
continuous streaming image. Szeliski et al discloses "presenting a series of individual video clips 
that are joined in to the appearance of a continuous streaming image" {lines 4-9 of column 13: 
"Rather, the video clip could be made up of multiple sequences of the scene captured at different 
times. Regardless of how many video sequences make up the inputted video clip, the trick is to 
produce the aforementioned new sequences such that the motion appears smooth and seamless 
to the viewer. ") of a character {lines 43-45 of column 2: "For example, another application of 
the video sprite concept involves objects that move about the scene in the input video clip such as 
an animal, vehicle, and person. "). 

29. Kanbe et al and Szeliski et al are analogous art because they are from the same problem 
solving area: simulating realistic interaction with computer animated objects. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to modify the 
characters disclosed by Kanbe et al with the video sprite disclosed by Szeliski et al in order to 
give the character the appearance of a continuous streaming image. The motivation for doing so 
would have been to achieve a greater degree of realism by giving the character "dynamic 
qualities" as stated by Szeliski et al in lines 9-1 1 of column 2. Therefore, it would have been 
obvious to combine Kanbe et al with Szeliski et al to obtain the invention as specified in claim 
11. 
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30. Claim 12 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of controlling the character in response to situations in a game" 
(machine translation of paragraph 72: "Drawing 7 is a flow chart which shows change item 
control routines, such as face expression change according to the waging-war situation for every 
CPU character by the mah-jongg game equipment of this invention. "). 

3 1 . Claim 15 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of using a series of triggers to control the character with responses that 
are different depending on game situation" (the triggers corresponding to character responses 
(facial expressions, voices and messages) are presented in machine translated paragraphs 63 
and 64.) For example, in items 3-7 of paragraph 63, Kanbe et al discloses a trigger ("the CPU 
character itself - reorganization - applying. . .") and varied character responses ("sadness face, 
"usually face", "resentment face") depending on the number of tiles. 

32. Claim 16 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of using a series of triggers to control the character with responses that 
are different depending on a game situation," as shown in the rejection of claim 15. Furthermore, 
Kanbe et al discloses that "the responses are continuous, wherein the responses include 
movement both during and between game moves of the character and the human operator" 
(machine translation of paragraph 138: "...changes every moment according to this invention as 
mentioned above — the expression of a face, a motion of a face, a message... ")• Furthermore, in 
paragraphs 62-64, Kanbe et al lists various game moves by the human player and other CPU 
controlled players that can occur between and during a human operator's turn, such as playing 
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tiles (machine translation of paragraph 62: "2. When the Last Wind Tile is Player (13-15, 
Resentment Face) "), that cause a change in the character's expression. 

33. Claim 18 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of using triggers to control responses of the character, wherein the 
triggers include the human operator's sequence of moves during a game" (machine translation 
of paragraph 42 gives the situations that change the feeling value, "reorganization can be 
applied at the time of the discarded tile of other houses (-ten points).,, "). Specifically, machine 
translation of paragraph 42 shows that the sequence of moves of other players (including those of 
the human operator) may result in a change in the feeling value of the character, which in turn 
triggers a response in a character as described in the rejection of claim 1 1 . 

34. Claim 19 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of using triggers to control responses of the character, wherein the 
triggers include the human operator's strategic position in a game" (machine translation of 
paragraph 42 as shown in the rejection of claim 18.) Kanbe et al does not explicitly use this 
language; however one of ordinary skill in the art would recognize that this feature is inherent 
because paragraph 42 shows that the feeling value of a computer opponent is decremented in 
response to a number of actions indicating the strategic position of the other players including 
the human operator. (The feeling value is used to determine a response in the character, such as a 
change in expression, as described in the rejection of claim 11.) 

35. Claim 23 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the character is an opponent in a game" (machine translation of paragraph 3 gives an 
overview of the game, where the office interprets the machine translated English phrase 
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"waging-war character" to be a description of an opponent character, as the invention clearly 
pertains to a well-known game "mah-jongg. ") 

36. Claim 28 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "presenting a two-dimensional representation of a playing area" {Drawing 2; machine 
translation of paragraph 27: "In the display screen as shown in drawing 2 as which the hand 
and the contents of a discarded tile of the player, the contents of a discarded tile of the waging- 
war character, a message, and face expression... "). 

37. With regard to claim 20, Kanbe et al discloses modifying the characters based on 
triggers, such as changing the expression based on the triggers as shown in claim 11. Kanbe et al 
does not specifically disclose interacting decision logic with a video controller. Szeliski et al 
discloses "interacting decision logic with a video controller" in Fig. 12 in order to schedule 
primitive video loops to create a new video sequences from fixed length video clips. 

38. With regard to claim 22, Kanbe et al discloses modifying the characters as shown in 
claim 11. Kanbe et al does not specifically disclose a video library. Szeliski et al discloses "the 
operation of modifying a character using a library of videos" {lines 33-36 of column 20: "After 
finding the list of primitive loops in the lowest cost compound loop for a particular loop length, 
the primitive loops (or transitions) are scheduled in some order so that they form a valid 
compound loop as described above. "; lines 43-45 of column 2: "For example, another 
application of the video sprite concept involves objects that move about the scene in the input 
video clip-such as an animal, vehicle, and person. "). Szeliski et al does not use the explicit 
language video libraries; however, one of ordinary skill in the art would recognize that the "list 
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of primitive fixed length video sequences" of lines 33-26 of column 20 are analogous to a library 
of videos as broadly recited in claim 22. 

39. With regard to claim 80, Kanbe et al discloses "presenting a background; setting a 
character against the background." Kanbe et al discloses in drawing 2 presenting a character 
represented by an image of a face contained in a rectangular region set against a background that 
serves as a representation of a playing area. Furthermore, Kanbe et al discloses "inputting a 
human operator's choice of action" (as shown in the rejection of the limitation of claim 1 1 
recited on lines 19 of page 13); "updating a current state based on the operator's action or 
inaction" (as shown in the rejection of the limitation of claim 1 1 recited on lines 20 of page 13); 
"using the current state by a decision logic to determine a response in a setting by the character" 
(as shown in the rejection of the limitation of claim 1 1 recited on lines 21-22 of page 13); 
"modifying the character" (as shown in the rejection of the limitation of claim 1 1 recited on 
lines 23 of page 13). With regard to the limitation of claim 80 recited on line 17 on page 23, 
Kanbe et al does not specifically disclose a video library. Szeliski et al discloses "the operation 
of modifying a character using video libraries" as shown in the rejection of claim 22. 

40. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a method for sequencing and transitioning between video sequences as 
disclosed by Szeliski et al to further modify the characters disclosed by Kanbe et al. The 
motivation for doing so would have been to achieve a greater degree of realism by giving the 
character (an animal, vehicle or person in a video game) "dynamic qualities" as stated by 
Szeliski et al in lines 9-1 1 of column 2. Therefore, it would have been obvious to further modify 
Kanbe et al with Szeliski et al to obtain the invention as specified in claims 20, 22, and 80. 
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41. With regard to claim 26, Kanbe et al shows the limitations of claim 1 1, but does not 
disclose a background by providing a series of concatenated still pictures. Szeliski et al discloses 
"the operation of presenting a digital background by providing a series of concatenated still 
pictures generated to provide life-like movement in the background" {lines 16-18 of column 2: 
"video textures could also find application as dynamic backdrops for scenes composite from live 
and synthetic elements"', lines 21-24 of column 3: "Further, the cost of transitioning between a 
particular frame and another frame is computed using the similarity between the next frame in 
the input video following the frame under consideration. "). Szeliski et al does not use the 
explicit language "concatenated still pictures"; however, one of ordinary skill in the art would 
recognize that this feature is inherent from the statement on lines 21-24 of column 3 that the 
frames of the video sequences are treated individually. Thus, the frames of the video sequence 
used for a background would be "a series of concatenated still-pictures." 

42. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to further modify the combination of Szeliski et al and Kanbe et al to include a 
background of concatenated still pictures. The motivation for doing so would have been to give 
the background "dynamic qualities" as stated by Szeliski et al in lines 9-1 1 of column 2, instead 
of using a static photo, thereby increasing the realism of the interaction. Therefore, it would have 
been obvious to further modify Kanbe et al with Szeliski et al to obtain the invention as specified 
in claim 26. 

43. Claim 30 is rejected with the rationale of claim 11. Claim 30 is similar in scope to claim 
11, wherein video clips are analogous to animation clips. 
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44. Claim 31 is met by the combination of Kanbe et al with Szeliski et al, wherein Szeliski et 
al discloses "the operation of modifying a character using a library of animations" (lines 33-36 of 
column 20: "After finding the list of primitive loops in the lowest cost compound loop for a 
particular loop length, the primitive loops (or transitions) are scheduled in some order so that 
they form a valid compound loop as described above. "; lines 43-45 of column 2: "For example, 
another application of the video sprite concept involves objects that move about the scene in the 
input video clip-such as an animal, vehicle, and person " ). Szeliski et al does not use the explicit 
language video libraries; however, one of ordinary skill in the art would recognize that the "list 
of primitive fixed length video sequences" of lines 33-26 of column 20 are analogous to a library 
of animation as broadly recited in claim 31 . 

45. With regard to claims 32 and 33, the Kanbe et al with Szeliski et al combination does not 
show that the animation is cell or clay animation; instead, the Kanbe et al and Szeliski et al 
combination shows the animation is from real-world video. 

46. At the time the invention was made, it would have been an obvious matter of design 
choice to a person of ordinary skill in the art to use clay animation or cell animation because the 
applicant has not disclosed that using clay animation or cell animation provides an advantage, is 
used for a particular purpose, or solves a stated problem. Furthermore, one of ordinary skill in 
the art would have expected the Kanbe et al and Szeliski et al combination to perform equally 
well with either real- world video animation as taught by Szeliski et al or the claimed clay (claim 
33) or cell (claim 32) animation because all three types of animations would perform the same 
function of representing a character in an interaction equally well. 
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47. Therefore, it would have been prima facie obvious to modify the Kanbe et al and Szeliski 
et al combination to obtain the invention as specified in claims 32 and 33 because such a 
modification would have been considered a mere design consideration which fails to patentably 
distinguish over the prior art of Kanbe et al and Szeliski et al. 

48. Claim 34 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et al 
discloses "the operation of using the current state by the decision logic to determine how to 
control the background in the environment" (drawing 2 shows the playing area which also serves 
as the background which displays the current state of the game; machine translation of 
paragraph 5: "The control section which obtains image data and/or voice data so that it may be 
in the display and/or output state according to the waging-war situation of said waging-war 
character... "). 

49. With regard to claim 87, Kanbe et al discloses "inputting an operator's choice of action 
or inaction" (as shown in the rejection of claim 11); "updating a current state based on the 
operator's action or inaction" (as shown in the rejection of claim 1 1); "using the current state by 
a decision logic to determine a response in a setting by the character" (as shown in the rejection 
of claim 11.) Kanbe et al does not disclose streaming video or video clips. Szeliski et al discloses 
"presenting a streaming video of a real-world background" (line 66 of column 1 through line 1 of 
column 2: "The new medium, which is referred to as a video texture, can provide a continuous, 
infinitely varying stream of video images. "; line 65 of column 21 through line 1 of column 22 as 
shown in the rejection of claim 11; lines 16-18 of column 2: "Video textures could also find 
application as dynamic backdrops for scenes composited from live and synthetic elements. "). 
Szeliski et al discloses, "presenting a series of individual video clips that are joined in to the 
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appearance of a continuous streaming image of a real world-character" (as shown in the rejection 
of claim 1 1). Szeliski et al does not use the explicit language "real-world character" in lines 4-9 
of column 13 and lines 43-45 of column 2; however, one of ordinary skill in the art would 
recognize that this feature is inherent from the statement on lines 14-16 of column 2: "Or an 
actor could provide a dynamic 'head shot' with continuous movement on his home page." 
Szeliski et al discloses "using the current state by the decision logic to determine a selection of 
how to control video of the character and how to control video of the background" {lines 4-9 of 
column 26: "As the user moves a slider (e.g., a time bar like on a video player) selecting a 
certain temporal portion of the video, the synthesis attempts to select frames that remain within 
that portion of the video, while at the same time using only fairly smooth transitions to jump back 
in time"). As previously shown, Szeliski et al discloses using a video texture as a background. 
50. With regard to claim 89, Kanbe et al discloses "the situation state of the digital 
environment," as shown in claim 11. Kanbe et al discloses "a course of action," as shown in 
claim 65. Szeliski et al discloses "a library of videos," as shown in claim 22 (incorporated here 
by reference. With regard to the limitation of claim 89 recited on line 28 of page 25 through line 
2 of page 26, Kanbe et al discloses "relating a current situation state and current triggers to a 
course of action to determine the most appropriate update to the environment" in machine 
translated paragraph 55, where the manner in which the tile is thrown is depended on the feeling 
value and the state of the game. Kanbe et al discloses that the feeling value is incremented and 
decremented based on triggers as shown in machine-translated paragraph 42. Kanbe et al does 
not show relating state and triggers to videos to determine the most appropriate update to the 
environment. Szeliski et al discloses "a video controller; and decision logic for relating current 
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situation state triggers to library of videos to determine a most appropriate update to the digital 
environment" {lines 4-9 of column 26: "As the user moves a slider (e.g., a time bar like on a 
video player) selecting a certain temporal portion of the video, the synthesis attempts to select 
frames that remain within that portion of the video, while at the same time using only fairly 
smooth transitions to jump back in time. "), and "for interacting with the video controller to 
modify a character and a background in the virtual reality environment presented on the display" 
in Fig. 12 in order to schedule primitive video loops to create a new video sequences from fixed 
length video clip. As previously shown in the rejection of claim 87 (the limitation stated on lines 
10-1 1 on page 25), Szeliski et al shows that the video texture can be used as a character or as a 
background and the videos will be controlled in an analogous manner in either embodiment. 

51. At the time of the invention, it would have been obvious to modify the Szeliski et al 
controlled character embodiment by substituting a background video instead of the character 
video used in the above embodiment disclosed by Szeliski et al, as suggested by Szeliski et al in 
lines 16-17 of column 2. Furthermore, at the time of the invention, it would have been obvious to 
a person of ordinary skill in the art to substitute a real-world background and character video as 
disclosed by Szeliski et al for the synthetic non- video background and character of Kanbe et al. 
The motivation for doing so would have been to give the character and the background "dynamic 
qualities" as stated by Szeliski et al in lines 9-1 1 of column 2, enhancing the realism of the 
interaction. Therefore, it would have been obvious to combine Kanbe et al with Szeliski et al to 
obtain the invention as specified in claims 87 and 89. 

52. Claim 90 is met by the combination of Kanbe et al and Szeliski et al, wherein Kanbe et 
al discloses "the operation of using triggers to control responses of the character, wherein the 
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triggers include the human operator's sequence of moves during a game" (machine translation of 
paragraph 42 gives the situations that change the feeling value, "reorganization can be applied 
at the time of the discarded tile of other houses (-ten points)... "), and Kanbe et al discloses "the 
operation of using triggers to control responses of the character, wherein the triggers include the 
human operator's sequence of moves during a game" (machine translation of paragraph 42 
gives the situations that change the feeling value, "reorganization can be applied at the time of 
the discarded tile of other houses (-ten points)... "). Specifically, Kanbe et al discloses in 
machine translation of paragraph 42 shows that the sequence of moves of other players 
(including those of the human operator) may result in a change in the feeling value of the 
character, which in turn triggers a response in a character as described in the rejection of claim 
11. 

53. Claims 13, 17, 21, 81, 82, and 91 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kanbe et al in view of Szeliski et al and in further view of U.S. Patent 
No. 6,798,426 to Tateishi (herein referred to as "TateishL") 

54. With regard to claim 13, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 1 1 on which claim 13 depends, but does not teach controlling the character 
"in response to time between moves by the human operator." Tateishi discloses "the operation of 
controlling the character in response to the time between moves by the human operator" (Fig. 
12; lines 64-67 of column 12: "A display image 9B shows the state wherein the state of no 
command input relative to the dialogue options displayed in front of the character has been 
repeated by given integer times. "). 



Application/Control Number: 1 0/623,034 Page 24 

Art Unit: 2671 

55. With regard to claim 17, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 1 1 on which claim 13 depends, and teaches "the operation of using a series 
of triggers for controlling the character in response to game situations, wherein game situations 
include actions by the human operator" as Kanbe et al discloses that a user's actions contribute to 
the game situation {machine translation of paragraph 70 : "That is, CPU 8 generates the 
command as a task for drawing or a voice output suitably based on the contents of directions 
(...a discarded tile, reach a tile, [other game conditions], etc.) directed from a game player 
through a controller 16. "), which subsequently triggers the character responses {machine 
translated paragraph 64). The combination of Kanbe et al and Szeliski et al does not include 
"inaction or time between moves" in the situation. Tateishi discloses "the operation of 
controlling the character in response to game situations including human operator inaction and 
the time between moves by the human operator" as shown in the rejection of claim 13. 

56. With regard to claim 91, the combination of Kanbe et al and Szeliski et al meet the 
limitations of claim 89 on which claim 91 depends, and the combination meets the limitation of 
claim 91 excluding the time between moves, wherein Kanbe et al discloses "the operation of 
using a series of triggers for controlling the character in response to game situations, wherein 
game situations include actions by the human operator," as shown in the rejection of claim 17. 
Tateishi discloses "the operation of controlling the character in response to game situations 
including human operator inaction and the time between moves by the human operator" as 
shown in the rejection of claim 13. 

57. Kanbe et al, Szeliski et al, and Tateishi are analogous art because they are from the same 
problem solving area: simulating realistic interaction with computer animated objects. At the 
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time of the invention it would have been obvious to a person of ordinary skill in the art to 
incorporate the time between the operator's moves as disclosed by Tateishi to the interaction 
between human and computer characters of Kanbe et al and Szeliski et al. The motivation for 
doing so would have been to make the computer-controlled characters seem more human to 
increase the realism of the interaction. Therefore, it would have been obvious to further modify 
Kanbe et al and Szeliski et al with Tateishi to obtain the invention as specified in claims 13, 17 
and 91. 

58. With regard to claim 21, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 1 1 on which claim 21 depends, but does not teach "making a direct change 
to the character." Tateishi discloses "the operation of modifying the character comprises making 
a direct change to the character" {SI 02 shown in Fig, 6; lines 1-4 of column 10: "In case of the 
manual designation (Yes at step SI 01), the player selects and sets a character and background 
data via a given menu image (step SI 02). "). 

59. At the time of the invention it would have been obvious to a person of ordinary skill in 
the art to add the operation of directly modifying the character as disclosed by Tateishi to game 
disclosed by Kanbe et al and Szeliski et al. The motivation for doing so would have been to 
allow the user to customize the interface, and provide variety to keep the user interested after 
several subsequent interactions. Therefore, it would have been obvious to further modify Kanbe 
et al and Szeliski et al with Tateishi obtain the invention as specified in claim 21. 

60. With regard to claim 81, the combination of Kanbe et al and Szeliski et al shows the 
limitations of claim 80 on which claim 81 depends. Furthermore, the combination of Kanbe et al 
and Szeliski et al meets the limitation of claim 81 recited on lines 20 and 21 of page 23, wherein 
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Kanbe et al discloses "using the current state by a decision logic to determine how to control the 
background in the environment," as shown in the rejection of claim 66. The combination of 
Kanbe et al and Szeliski et al meets the limitation of claim 81 recited on lines 22 and 23 of page 
23, wherein Szeliski et al discloses "interacting decision logic with a video controller" in Fig. 12 
in order to schedule primitive video loops to create a new video sequences from fixed length 
video clips. The combination of Kanbe et al and Szeliski et al does not teach "sending a selection 
of control of the background to the display." Tateishi discloses "the operation of sending a 
selection of control of the background to the display" {SI 02 shown in Fig. 6; lines 1-4 of column 
10: ' 'In case of the manual designation (Yes at step SI 01), the player selects and sets a character 
and background data via a given menu image (step SI 02). "). 

61 . At the time of the invention it would have been obvious to a person of ordinary skill in 
the art to add the operation of controlling the background and a character as disclosed by Tateishi 
to game disclosed by Kanbe et al and Szeliski et al. The motivation for doing so would have 
been to allow the user to customize the interface, and provide variety to keep the user interested 
after several subsequent interactions. Therefore, it would have been obvious to further modify 
Kanbe et al and Szeliski et al with Tateishi obtain the invention as specified in claim 81. 

62. Claim 82 is met by the combination of Kanbe et al, Szeliski et al and Tateishi, wherein 
Szeliski et al discloses "presenting a continuously streaming video" (line 66 of column 1 through 
line 1 of column 2: "The new medium, which is referred to as a video texture, can provide a 
continuous, infinitely varying stream of video images. "; lines 51-52 of column 2: "Further, the 
frames of the video sprite could be inserted into a previously derived background image (or 
frames of a background video)... 
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63. Claims 27, 29, 35, 83, 86, and 88 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kanbe et al in view of Szeliski et al and in further view of U.S. Patent 
No. 6,676,518 to Sawa et al (herein referred to "Sawa et al.") 

64. With regard to claim 27, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 1 1 on which claim 27 depends, but does not specifically show a three- 
dimensional representation of a playing area. Sawa et al discloses a "three-dimensional 
representation of a playing area" (Figs, 5-8, and 14-15 show a three-dimensional representation 
31 of a soccer playing area. "). 

65. With regard to claim 29, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 1 1 on which claim 29 depends, but does not specifically show 
simultaneously presenting a two-dimensional and three-dimensional area simultaneously. Sawa 
et al shows "presenting a two-dimensional and three-dimensional area simultaneously" (Figs. 5- 
8, and 14-15 show a three-dimensional representation of a soccer playing area 31 and the 
corresponding two-dimensional view 36 simultaneously.). 

66. With regard to claim 35, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 30 on which claim 35 depends, and meets the limitation of claim 35 recited 
on line 28 of page 16, wherein Kanbe discloses a "two-dimensional representation of the playing 
area" (drawing 3.) The combination of Kanbe et al and Szeliski et al does not disclose a three- 
dimensional representation of a playing area. Sawa et al discloses a "three-dimensional 
representation of a playing area" (Figs. 5-8, and 14-15 show a three-dimensional representation 
31 of a soccer playing area. "). Sawa et al discloses "simultaneously presenting a two-dimension 
representation of the playing area to transmit information regarding a game to an operator and to 
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permit the operator to simultaneously observe action in both two and three dimensions" (Fig. 5 
shows a three-dimensional view of a character 32 in the three-dimensional view 31 and the 
character 32 at the corresponding position in the two-dimensional view 36; lines 25-28 of 
column 5: "In this map 36, the positions of the player characters of the own and enemy teams 
and that of the player character 32 are distinguishably displayed. "). 

67. With regard to claim 86, the combination of Kanbe et al and Szeliski et al meets the 
limitations of claim 85 on which claim 86 depends, but does not disclose a three-dimensional 
representation of a playing area. Sawa et al discloses a "three-dimensional representation of a 
playing area," as shown in the rejection of claim 35. Sawa et al discloses "simultaneously 
presenting a two-dimension representation of the playing area to transmit information regarding 
a game to an operator and to permit the operator to simultaneously observe action in both two 
and three dimensions," as shown in the rejection of claim 35, wherein Sawa et al discloses the 
game is a soccer game. 

68. With regard to claim 88, the Kanbe et al and Szeliski et at combination meets the 
limitations of claim 87 on which claim 88 depends, and the limitation of claim 88 recited on lines 
18 and 19 of page 25, wherein Kanbe et al discloses "the character's response is a response in a 
game, and wherein the character is a computer controlled opponent in the game" (machine 
translation of paragraph 72: "Drawing 7 is a flow chart which shows change item control 
routines, such as face expression change according to the waging-war situation for every CPU 
character by the mah-jongg game equipment of this invention. "). As shown in the rejection of 
claim 11, Kanbe et al discloses that the computer controlled character responds in a number of 
ways in response to the mah-jongg game. The Kanbe et al and Szeliski et al combination does 
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not show a three-dimensional representation of a playing area. Sawa et al discloses "presenting a 
three-dimensional representation of a playing area, and simultaneously presenting a two- 
dimensional representation of the playing area to transmit information regarding a game to an 
operator and to permit the operator to simultaneously observe action in both two and three 
dimensions" as shown in the rejection of claim 35. 

69. Kanbe et al, Szeliski et al, and Sawa et al are analogous art because they are from the 
same problem solving area: simulating realistic interaction with computer animated objects. At 
the time of the invention, it would have been obvious to a person of ordinary skill in the art to 
use the method taught by Sawa et al to present the playing area of Kanbe et al and Szeliski et al. 
The motivation for doing so would have been to provide both a realistic view and a 
comprehensive overview of the current state of the game. Therefore, it would have been obvious 
to further modify Kanbe et al and Szeliski et al with Sawa et al to obtain the invention as 
specified in claims 27, 29, 35, 89, and 88. 

70. With regard to claim 83, the combination of Kanbe et al, Szeliski et al and Tateishi meets 
the limitation of claim 80 on which claim 83 depends. The limitation recited on line 2 of page 24 
is met by the combination Kanbe et al, Szeliski et al and Tateishi, wherein Kanbe et al discloses 
"presenting a two-dimensional representation of a playing area" (Drawing 2; machine 
translation of paragraph 27: "In the display screen as shown in drawing 2 as which the hand 
and the contents of a discarded tile of the player, the contents of a discarded tile of the waging- 
war character, a message, and face expression,.. "). The Kanbe et al, Szeliski et al and Tateishi 
combination does not show a three-dimensional representation. Sawa et al discloses a "three- 
dimensional representation of a playing area" (Figs. 5-8, and 14-15 show a three-dimensional 
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representation 31 of a soccer playing area. "). Sawa et al shows "presenting a two-dimensional 
and three-dimensional area simultaneously" (Figs. 5-8, and 14-15 show a three-dimensional 
representation of a soccer playing area 31 and the corresponding two-dimensional view 36 
simultaneously.). 

71 . Kanbe et al, Szeliski et al, Tateishi et al and Sawa et al are analogous art because they are 
from the same problem solving area: simulating realistic interaction with computer animated 
objects. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the method taught by Sawa et al to present the playing area of Kanbe et al, Szeliski 
et al and Tateishi et al. The motivation for doing so would have been to provide both a realistic 
view and a comprehensive overview of the current state of the game. Therefore, it would have 
been obvious to further modify Kanbe et al and Szeliski et al with Sawa et al to obtain the 
invention as specified in claim 83. 

72. Claims 8, 10, 36, 37, 42, 47, 51-56, 64, 65, 66, 68, 70, 73-79, 84, and 85 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Kanbe et al in view of Sawa et al. 

73. With regard to claims 8 and 10, Kanbe et al does not disclose a three-dimensional 
playing area. Sawa et al discloses "presenting a three-dimensional representation of a playing 
area; and simultaneously presenting a two-dimensional playing area" (Figs. 5-8, and 14-15 show 
a three-dimensional representation of a soccer playing area 31 and the corresponding two- 
dimensional view 36 simultaneously.) 

74. With regard to claims 36 and 37, Kanbe discloses a "two-dimensional representation of 
the playing area" (drawing 3.) Kanbe et al does not disclose a three-dimensional representation 
of a playing area. Sawa et al discloses a "three-dimensional representation of a playing area" 
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(Figs. 5-8, and 14-15 show a three-dimensional representation 31 of a soccer playing area. ")• 
Sawa et al discloses "wherein the three-dimensional representation and the two-dimensional 
representation are simultaneously presented on a single video screen to allow the human operator 
to simultaneously observe action in both two and three dimensions," as in claim 36 and "the 
three-dimensional view is a side view of the playing area as would be seen in real life," and the 
two-dimensional view is a top-down view," as in claim 37 (Fig, 5 shows a three-dimensional 
view of a character 32 in the three-dimensional side view 31 and the character 32 at the 
corresponding position in the two-dimensional top-down view 36, all presented on the same 
screen 2; lines 25-28 of column 5: "In this map 36, the positions of the player characters of the 
own and enemy teams and that of the player character 32 are distinguishably displayed. 
75. With regard to claim 42, Kanbe et al discloses a character along with a two-dimensional 
representation of a playing area (drawing 2) With regard to the limitations of claim 42 recited on 
lines 7 and 8 of page 18, Kanbe et al discloses in drawing 2 presenting a character represented by 
an image of a face contained in a rectangular region set against a background that serves as a 
representation of a playing area. Kanbe et al does not disclose the limitation of claim 42 on line 9 
("presenting a three-dimensional representation of a playing area") and the limitation of claim 42 
on lines 1 1 and 12 ("wherein the three-dimensional representation of the playing area and the 
two-dimensional representation of the playing area are presented simultaneously.") Sawa et al 
shows "presenting a three-dimensional representation of a playing area wherein the three- 
dimensional representation of the playing area and the two-dimensional representation of the 
playing area are presented simultaneously" (Figs. 5-8, and 14-15 show a three-dimensional 
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representation of a soccer playing area 31 and the corresponding two-dimensional view 36 
simultaneously.) 

76. With regard to claim 76, Kanbe et al discloses "the two-dimensional representation of the 
playing area of the playing area include the positions of pieces on a playing board representing 
the state of the game" (drawing 4 shows the mah-jongg playing board with the playing tiles 
positioned in the center and along the edge; machine translated paragraph 5: "The control 
section which obtains image data and/or output state according to the waging-war situation of 
said waging-war character. "). Kanbe et al does not disclose a three-dimensional representation 
of the playing area. Sawa et al discloses "and the three-dimensional representation of the playing 
area representing the state of the game" (Fig. 2 shows a three-dimensional view of the characters 
the positions of the characters on the playing area and a two-dimensional view; lines 25-28 of 
column 5: "In this map 36, the positions of the player characters of the own and enemy teams 
and that of the player character 32 are distinguishably displayed. "). 

77. Kanbe et al and Sawa et al are analogous art because they are from the same problem 
solving area: simulating realistic interaction with computer animated objects. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to use the method 
taught by Sawa et al to present the playing area and playing pieces of Kanbe et al. The 
motivation for doing so would have been to provide both a realistic view and a comprehensive 
overview of the current state of the game. Therefore, it would have been obvious to modify 
Kanbe et al with Sawa et al to obtain the invention as specified in claims 8, 10, 36, 37, 42 and 
76. 
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78. Claim 47 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of controlling the character in response to actions of a human operator" 
in machine translated paragraphs 64 and 70. Kanbe et al does not use this language explicitly; 
however, one of ordinary skill in the art would recognize this feature is inherent from the 
statements where Kanbe et al discloses that a user's actions contribute to the game situation 
{machine translation of paragraph 70: 'That is, CPU 8 generates the command as a task for 
drawing or a voice output suitably based on the contents of directions (...a discarded tile, reach 
a tile, [other game conditions], etc.) directed from a game player through a controller 16. "), 
which subsequently triggers the character responses {machine translated paragraph 64.) 

79. Claim 51 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of affecting the situation state of the environment responsive to an action 
or to inaction by the human operator" wherein the operator's actions, such as discarding a tile, 
can cause an update to the mah-jongg game state as shown in machine translated paragraphs 29 
and 94. 

80. Claim 52 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using a series of triggers to control the character with responses that 
are different depending on game situation" {the triggers corresponding to character responses 
(facial expressions, voices and messages) are presented in machine translated paragraphs 63 
and 64.) For example, in items 3-7 of paragraph 63, Kanbe et al discloses a trigger ("the CPU 
character itself - reorganization - applying. . .") and varied character responses ("sadness face, 
"usually face", "resentment face") depending on the number of tiles. 
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81. Claim 53 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using a series of triggers to control the character with responses that 
are different depending on a game situation," as shown in the rejection of claim 15. Furthermore, 
Kanbe et al discloses that "the responses are continuous, wherein the responses include 
movement both during and between game moves of the character and the human operator" 
{machine translation of paragraph 138: "...changes every moment according to this invention as 
mentioned above — the expression of a face, a motion of a face, a message... "). Furthermore, in 
paragraphs 62-64, Kanbe et al lists various game moves by the human player and other CPU 
controlled players that can occur between and during a human operator's turn, such as playing 
tiles (machine translation of paragraph 62: "2. When the Last Wind Tile is Player (13-15, 
Resentment Face) "), that cause a change in the character's expression. 

82. Claim 54 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using triggers to control responses of the character, wherein the 
triggers include the human operator's sequence of moves during a game" (machine translation 
of paragraph 42 gives the situations that change the feeling value, "reorganization can be 
applied at the time of the discarded tile of other houses (-ten points)... "). Specifically, machine 
translation of paragraph 42 shows that the sequence of moves of other players (including those of 
the human operator) may result in a change in the feeling value of the character, which in turn 
triggers a response in a character as described in the rejection of claim 11. 

83. Claim 55 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using triggers to control responses of the character, wherein the 
triggers include the human operator's strategic position in a game" (machine translation of 
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paragraph 42 as shown in the rejection of claim 18.) Kanbe et al does not explicitly use this 
language; however one of ordinary skill in the art would recognize that this feature is inherent 
because paragraph 42 shows that the feeling value of a computer opponent is decremented in 
response to a number of actions indicating the strategic position of the other players including 
the human operator, and the feeling value is used to determine a response in the character, such 
as a change in expression, as described in the rejection of claim 11. 

84. Claim 56 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using a current situation state of the interaction to determine a 
response in the environment by the character." Specifically, Kanbe et al discloses that the 
computer-controlled players respond to the current state in a number of ways: giving messages 
(machine translated paragraph 15), facial expressions (machine translated paragraph 7), and 
game actions in general such as discarding tiles (machine translated paragraph 5). Furthermore, 
Kanbe et al discloses, "the character is a computer-controlled player in a game" {machine 
translation of paragraph 3 gives an overview of the game, where the office interprets the 
machine translated English phrase "waging-war character" to be a description of an opponent 
character, as the invention clearly pertains to a well-known game "mah-jongg"; and the phrase 
"CPU players" to be computer controlled, which is commonly used in the art) 

85. Claim 64 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses 

i. "inputting a human operator's choice of action or the operator's inaction to a 
situation state module" using the controller 16 (machine translation of paragraph 70: 
"That is, CPU 8 generates the command as a task for drawing or a voice output suitably 
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based on the contents of directions (...a discarded tile, reach a tile... etc.) directed from a 
game player through a controller 16. "); 

j. "updating a current state based on the operator's action or inaction," wherein the 
operator's actions, such as discarding a tile, can cause an update to the mah-jongg game 
state as shown in machine translated paragraphs 29 and 94; 
k. "using the current state by a decision logic to determine a response in the 
environment by the character, wherein the character is a computer-controlled player," 
{machine translation of paragraph 1 7: "Furthermore, a motion of the face for every 
waging-war character as image data in this invention is a motion which looks at which 
direction at least among the direction into which the tile was thrown away, the direction 
of the partner who cried, and the direction of a partner which carried out win 
preferably. "; machine translation of paragraph 75: "As shown in drawing 12 , by 
making the example of the CPU character into Uchida (woman), first, a control section 
20 is controlled by the step ST 11 with game data, and the initial set (128 points) of the 
basic feeling value classified by CPU character is performed. "). As previously shown, 
the computer-controlled players respond to the current state in a number of ways: giving 
messages {machine translated paragraph 75), facial expressions {machine translated 
paragraph 7), and game actions in general such as discarding tiles {machine translated 
paragraph 5). 

86. Claim 65 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of putting the operator's course of action in place and sending it to a 
display." Kanbe et al discloses that the operators course of action is inputted using controller 16, 
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and lists of possible courses of action: "a discarded tile, reach a tile," (machine translation of 
paragraph 70); in the following paragraph, Kanbe et al discloses the course of action is 
displayed (machine translation of paragraph 71: "Then, the image drawing processing 
processor 12 performs write-in processing of the image data which should draw on RAM 8 etc. 
based on the above-mentioned count result "). 

87. Claim 66 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of using the current state by the decision logic to determine how to 
control the character" (items 3-7 of paragraph 63 shows character responses ("sadness face, 
"usually face", "resentment face") controlled depending on the number of tiles) "and the 
background in the environment" (drawing 2 shows the playing area which also serves as the 
background which displays the current state of the game; machine translation of paragraph 5: 
"The control section which obtains image data and/or voice data so that it may be in the display 
and/or output state according to the waging-war situation of said waging-war character... "). 

88. Claim 68 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "inputting a human operator's choice of action or the operator's inaction to a situation 
state module" (as shown in the rejection of claim 64 with regard to the limitation on lines 2 and 
3 on page 21); "updating a current state based on the operator's action or inaction" (as shown in 
the rejection of claim 64 with regard to the limitation on line 4 on page 21); "using the current 
state by a decision logic to determine how to control the background in the environment" (as 
shown in the rejection of claim 66). 

89. Claim 70 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of presenting courses of action to the display after utilizing appropriate 
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decision logic, wherein the decision logic is a computer algorithm" (machine translated 
paragraph 31: "Here, actuation by this invention is explained in more detail below about the 
control section 20 including CPU 6 controlled by the game data recorded on the record medium 
5 machine translated paragraph 39: "Next, the example is explained in detail about the 
concrete contents of the game data which consist of image data, voice data, program data, etc. "; 
drawing 1). 

90. Claim 73 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the operation of providing information regarding the current situation state on a 
display'' (machine translation of paragraph 29: "...the mah-jongg waging-war situation for 
every CPU character will be displayed on a screen... "). 

91 . Claim 74 is met by the combination of Kanbe et al and Sawa et al, wherein Kanbe et al 
discloses "the playing area includes a playing board, and wherein the playing area further 
includes playing pieces" (drawing 4 shows the mah-jongg playing board with the playing tiles 
positioned in the center and along the edges). 

92. Claim 75 is met by the combination of Kanbe et al and Sawa et al, wherein Sawa et al 
discloses "the two-dimensional representation of the playing area and the three-dimensional 
representation of the playing area include the state of the interaction" (Fig. 2 shows a three- 
dimensional view of the characters the positions of the characters on the playing area and a two- 
dimensional view; lines 25-28 of column 5: "In this map 36, the positions of the player 
characters of the own and enemy teams and that of the player character 32 are distinguishably 
displayed. "). 
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93. Claim 77 is met by the combination of Kanbe et al and Sawa et al, wherein Sawa et al 
discloses "the three-dimensional representation of the playing area is a view as would be seen in 
real-life from a player's perspective." Sawa et al shows in Fig. 5 a view of the playing area from 
a player's perspective in real-life, as evidenced by correct depth-cues such as player 34 
appearing smaller than player 32 and drawn at a higher position on the screen. 

94. Claim 78 is met by the combination of Kanbe et al and Sawa et al, wherein Sawa et al 
discloses "the two-dimensional representation of the playing area is a top-down view of the 
playing area, and is positioned in front of the background, and is transparent" (a two-dimensional 
view is a top down view of the playing area 31 is shown in Figs. 2, 5-8, 14, 15). 

95. Claim 79 is met by the combination of Kanbe et al and Sawa et al, wherein Sawa et al 
discloses "the two-dimensional representation of the playing area shows all corresponding 
movement in both time and space from a game shown in the three-dimensional representation" 
{Figs. 2, 5-8, 14, 15 shows a player 32 at the same position with respect to other features in the 
3-dimensional view on the playing area with in the three-dimensional view 31 and the two- 
dimensional view 36; lines 25-28 of column 5: "In this map 36, the positions of the player 
characters of the own and enemy teams and that of the player character 32 are distinguishably 
displayed. "). 

96. With regard to claim 84, Kanbe et al discloses "inputting an operator's choice of action 
or inaction" using the controller 16 {machine translation of paragraph 70: "That is, CPU 8 
generates the command as a task for drawing or a voice output suitably based on the contents of 
directions (...a discarded tile, reach a tile, [other game actions], etc.) directed from a game 
player through a controller 16. "); "updating a current state based on the operator's action or 
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inaction; using the current state by a decision logic to determine a response in a setting by the 
character; modifying the character," (as shown in the rejection of claim 11 of the limitation 
recited on lines 19-24 on page 13). Kanbe et al does not disclose a streaming video or a series of 
video clips as in the limitation recited on lines 8 and 9 on page 24. Szeliski et al discloses 
"presenting a series of individual video clips that are joined in to the appearance of a continuous 
streaming image" (lines 4-9 of column 13: "Rather, the video clip could be made up of multiple 
sequences of the scene captured at different times. Regardless of how many video sequences 
make up the inputted video clip, the trick is to produce the aforementioned new sequences such 
that the motion appears smooth and seamless to the viewer. ") of a real-world character (lines 43- 
45 of column 2: "For example, another application of the video sprite concept involves objects 
that move about the scene in the input video clip such as an animal, vehicle, and person. "). 
Szeliski et al discloses "presenting a streaming video of a real-world background scene" (line 66 
of column 1 through line 1 of column 2: "The new medium, which is referred to as a video 
texture, can provide a continuous, infinitely varying stream of video images. "; line 65 of column 
21 through line 1 of column 22 as shown in the rejection of claim 11; lines 16-18 of column 2 as 
shown in the rejection of claim 71). 

97. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the character motion and the background disclosed by Kanbe et al with the 
method for creating a continuous character and background motion from video disclosed by 
Szeliski et al. The motivation for doing so would have been to achieve a greater degree of 
realism by giving a character and the background "dynamic qualities," as suggested by Szeliski 
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et al in lines 9-11 of column 2. Therefore, it would have been obvious to further modify the 
combination of Kanbe et al with Szeliski et al to obtain the invention as specified in claim 84. 

98. With regard to claim 85, the limitations of the parent claim 84 are met by the of Kanbe et 
al and Szeliski et al. Furthermore, Kanbe et al discloses "using the current state by a decision 
logic to determine how to control the background in the environment" (as shown in the rejection 
of claim 66). Szeliski et al discloses "interacting decision logic with a video controller" in Fig. 
12 in order to schedule primitive video loops to create a new video sequences from fixed length 
video clips. 

99. In an embodiment, Szeliski shows "interacting decision logic with a video controller to 
modify the background after the decision logic uses the current state" lines 4-9 of column 26: 
"As the user moves a slider (e.g., a time bar like on a video player) selecting a certain temporal 
portion of the video, the synthesis attempts to select frames that remain within that portion of the 
video, while at the same time using only fairly smooth transitions to jump back in time." As 
previously shown in the rejection of parent claim 84, Szeliski et al discloses using video textures 
as background. 

100. At the time of the invention it would have been obvious to a person of ordinary skill in 
the art to further modify the Kanbe et al and Szeliski et al combination to substitute video 
coupled with decision logic to change the background based on the current state as disclosed by 
Szeliski et al instead of traditional animation as disclosed by Kanbe et al. The motivation for 
doing so is stated by Szeliski et al on lines 33-35 of column 29: "The user-directed video sprites 
are an alternative to traditional animation where using easily and cheaply acquired video footage 
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makes elaborate hand modeling unnecessary." Therefore, it would have been obvious to further 
modify Kanbe et al with Szeliski et al to obtain the invention as specified in claim 85. 

101. Claim 38 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kanbe et al 
in view of Sawa et al and in further view of U.S. Patent 6,784,901 to Harvey et al (herein 
referred to as "Harvey et al.") 

102. With regard to claim 38, the combination of Kanbe et al and Sawa et al shows a two- 
dimensional view. Kanbe et al and Sawa et al does not specifically disclose a see-through view. 
Harvey et al discloses "a two-dimensional window overlaid on a three-dimensional view that is 
see-through" (Fig. 11; lines 46-49 of column 9: 'Items window 1116 may be translucent so as to 
not completely obscure the action in the 3D world. "). 

103. Kanbe et al, Sawa et al and Harvey et al are analogous art because they are from the same 
problem solving area: simulating a realistic interaction. At the time of the invention it would 
have been obvious to a person of ordinary skill in the art to modify the combination of Kanbe et 
al and Sawa et by making the two-dimensional view see-though as taught by Harvey et al. The 
motivation for doing so would have been to "not completely obscure the action in the 3D world," 
as stated by Harvey et al in lines 46-49 of column 9. Therefore, it would have been obvious to 
further modify Kanbe et al and Sawa et al with Harvey et al to obtain the invention as specified 
in claim 38. 

104. Claims 39 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kanbe et al in view of Sawa et al and Harvey et al and in further view of Szeliski et al. 

105. With regard to claim 39, the combination of Kanbe et al, Sawa et al and Harvey et al 
meets the limitation of claim 38 on which claim 39 depends, and meets the limitation of claim 39 
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recited on line 22 of page 17, wherein Kanbe et al discloses "setting a character against the 
background as shown in the rejection of limitations of claim 42 recited on lines 7 and 8 of page 
18. The combination of Kanbe et al, Sawa et al and Harvey et al does not show video. Szeliski et 
al discloses "continuously streaming video of a background scene" {line 66 of column 1 through 
line 1 of column 2: "The new medium, which is referred to as a video texture, can provide a 
continuous, infinitely varying stream of video images. "; line 65 of column 21 through line 1 of 
column 22 as shown in the rejection of claim 11; lines 16-18 of column 2: "Video textures could 
also find application as dynamic backdrops for scenes composited from live and synthetic 
elements. "). 

106. With regard to claim 40, the combination of Kanbe et al, Sawa et al and Harvey et al 
does not disclose a series of video clips. Szeliski et al discloses "presenting a series of individual 
video clips that are joined in to the appearance of a continuous streaming image" {lines 4-9 of 
column 13: "Rather, the video clip could be made up of multiple sequences of the scene captured 
at different times. Regardless of how many video sequences make up the inputted video clip, the 
trick is to produce the aforementioned new sequences such that the motion appears smooth and 
seamless to the viewer. ") of a character {lines 43-45 of column 2: "For example, another 
application of the video sprite concept involves objects that move about the scene in the input 
video clip such as an animal, vehicle, and person. "). 

107. Kanbe et al, Sawa et al, Harvey et al, and Szeliski et al are analogous art because they are 
from the same problem solving area: simulating realistic interaction with virtual objects. At the 
time of the invention it would have been obvious to a person of ordinary skill in the art to 
substitute a continuous video background and character as disclosed by Szeliski et al for the 
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synthetic non-video background and character of the Kanbe et al, Sawa et al, and Harvey et al 
combination. The motivation for doing so would have been to give the character and the 
background "dynamic qualities" as stated by Szeliski et al in lines 9-1 1 of column 2, enhancing 
the realism of the interaction. Therefore, it would have been obvious to combine Kanbe et al, 
Sawa et al, and Harvey et al with Szeliski et al to obtain the invention as specified in claims 39 
and 40. 

108. Claim 41 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kanbe et al 
in view of Sawa et al, Harvey et al, Szeliski et al in further view of Linnett et al. 

109. With regard to claim 41, the combination of Kanbe et al, Sawa et al, Harvey et al, and 
Szeliski et al meets all of the limitations of claim 40 on which claim 41 depends, but does not 
show a time-depended motion of the character based on user interaction. Linnett et al discloses 
"providing a motion of the character in a time-dependent manner based on interaction with the 
human operator" (lines 38-41 of column 10: 'The idle animation is displayed whenever no other 
commands have been send to the actor services 112. It should be appreciated that the idle 
animation may be requested by applications 28 and by the shell 30. "). 

110. Kanbe et al, Sawa et al, Harvey et al, Szeliski et al and Linnett et al are analogous art 
because they are from the same problem solving area: simulating realistic interaction with virtual 
objects. At the time of the invention it would have been obvious to a person of ordinary skill in 
the art to incorporate the aspect of time with respect to character motion as disclosed by Linnett 
et al to the interaction between human and computer characters of the Kanbe et al, Sawa et al, 
Harvey et al, and Szeliski et al combination. The motivation for doing so would have been to 
make the computer-controlled characters seem more human to increase the realism of the 
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interaction. Therefore, it would have been obvious to further modify the Kanbe et al, Sawa et al, 
Harvey et al, and Szeliski et al combination with Linnett to obtain the invention as specified in 
claim 41. 

111. Claims 43-46, 57, 59, 69, 71, and 72 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kanbe et al in view of Sawa et al and in further view of Szeliski et al. 

112. With regard to claim 43, the combination of Kanbe et al and Sawa et al show the 
limitations of claim 42, and Kanbe et al discloses character motion {machine translated 
paragraph 38: "Moreover, a motion of the face for every waging-war character carries out the 
motion which looks at which direction... "), and "the character is a representation of a real-world 
potential player" (machine translation of paragraph 3 gives an overview of the game, where the 
office interprets the machine translated English phrase "waging-war character" to be a 
description of an opponent character, as the invention clearly pertains to a well-known game 
"mah-jongg. "). The combination of Kanbe et al and Sawa et al does not show "presenting a 
continuously streaming video of the character." Szeliski et al discloses "presenting a 
continuously streaming video" (line 66 of column 1 through line 1 of column 2: "The new 
medium, which is referred to as a video texture, can provide a continuous, infinitely varying 
stream of video images. "; lines 51-52 of column 2: "Further, the frames of the video sprite could 
be inserted into a previously derived background image (or frames of a background video)... ") 
of a character (lines 43-45 of column 2: "For example, another application of the video sprite 
concept involves objects that move about the scene in the input video clip such as an animal, 
vehicle, and person. "). 
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113. With regard to claim 45, the combination of Kanbe et al and Sawa et al show the 
limitations of claim 42, and Kanbe et al discloses character motion as shown in the rejection of 
claim 43. The combination of Kanbe et al and Sawa et al does not specifically teach using "video 
clips joined into the appearance of a continuous streaming image of a character." Szeliski et al 
discloses "presenting a series of individual video clips that are joined in to the appearance of a 
continuous streaming image" {lines 4-9 of column 13: "Rather, the video clip could be made up 
of multiple sequences of the scene captured at different times. Regardless of how many video 
sequences make up the inputted video clip, the trick is to produce the aforementioned new 
sequences such that the motion appears smooth and seamless to the viewer. ") of a character 
(lines 43-45 of column 2: "For example, another application of the video sprite concept involves 
objects that move about the scene in the input video clip such as an animal, vehicle, and 
person. "). 

114. With regard to claim 57, the combination of Kanbe et al and Sawa et al show modifying 
a character: Kanbe et al discloses modifying the characters based on triggers, such as changing 
the character's expression as previously shown. Kanbe et al does not specifically teach 
interacting decision logic with a video controller. Szeliski et al discloses "interacting decision 
logic with a video controller" in Fig. 12 in order to schedule primitive video loops to create a 
new video sequences from fixed length video clips. 

115. With regard to claim 59, as previously shown, the combination of Kanbe et al and Sawa 
et al show modifying a character. The combination of Kanbe et al and Sawa et al does not 
specifically teach a video library. Szeliski et al discloses "the operation of modifying a character 
using a library of videos" (lines 33-36 of column 20: "After finding the list of primitive loops in 
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the lowest cost compound loop for a particular loop length, the primitive loops (or transitions) 
are scheduled in some order so that they form a valid compound loop as described above. "; 
lines 43-45 of column 2: "For example, another application of the video sprite concept involves 
objects that move about the scene in the input video clip-such as an animal, vehicle, and 
person "). Szeliski et al does not use the explicit language "a library of videos"; however, one of 
ordinary skill in the art would recognize that the "list of primitive fixed length video sequences" 
of lines 33-26 of column 20 are analogous to a library of videos as broadly recited in claim 59. 

1 16. Kanbe et al, Sawa et al, and Szeliski et al are analogous art because they are from the 
same problem solving area: simulating realistic interaction with computer animated objects. At 
the time of the invention, it would have been obvious to a person of ordinary skill in the art to 
modify the character motion disclosed by Kanbe et al with the method for creating a continuous 
character motion from video disclosed by Szeliski et al. The motivation for doing so would have 
been to achieve a greater degree of realism by giving the character "dynamic qualities" as stated 
by Szeliski et al in lines 9-1 1 of column 2. Therefore, it would have been obvious to further 
modify the combination of Kanbe et al and Sawa et al with Szeliski et al to obtain the invention 
as specified in claims 43, 45, 57, and 59. 

117. Claim 44 is rejected with the rationale of claim 43. Claim 44 is similar in scope to claim 
43. 

118. Claim 46 is rejected with the rationale of claim 45. Claim 46 is similar in scope to claim 
45. 

1 19. With regard to claim 71, as shown in the rejection of claim 57 the combination of Kanbe 
et al, Sawa et al and Szeliski et al shows "the operation of interacting with a video controller to 
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modify the character after utilizing appropriate decision logic." The combination of Kanbe et al 
and Sawa et al as modified by Szeliski et al in the rejection above does not show doing the same 
for a background. Szeliski et al discloses using the video texture, which is dependent on "the 
operation of interacting with a video controller" (as described above), as a background (lines 16- 
18 of column 2: "video textures could also find application as dynamic backdrops for scenes 
composite from live and synthetic elements 1 *). 

120. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the combination of Kanbe et al and Sawa et al so that the background is 
controlled by a video controller and decision logic as disclosed by Szeliski et al. The motivation 
for doing so would have been to achieve a greater degree of realism by giving a character and the 
background "dynamic qualities," as suggested by Szeliski et al in lines 9-1 1 of column 2. 
Therefore, it would have been obvious to further modify the combination of Kanbe et al and 
Sawa et al with Szeliski et al to obtain the invention as specified in claim 71. 

121. With regard to claim 72, the combination of Kanbe et al and Sawa et al does not disclose 
the limitation "the operation of continuously streaming video to present a background." Szeliski 
et al discloses "the operation of continuously streaming video to present a background" (line 66 
of column 1 through line 1 of column 2: "The new medium, which is referred to as a video 
texture, can provide a continuous, infinitely varying stream of video images. line 65 of column 
21 through line 1 of column 22 as shown in the rejection of claim 11; lines 16-18 of column 2 as 
shown in the rejection of claim 71). 

122. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the combination of Kanbe et al and Sawa et al so that the background is 
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continuously streaming video as disclosed by Szeliski et al. The motivation for doing so would 
have been to achieve a greater degree of realism by giving a character and the background 
"dynamic qualities," as suggested by Szeliski et al in lines 9-11 of column 2. Therefore, it would 
have been obvious to further modify the combination of Kanbe et al and Sawa et al with Szeliski 
et al to obtain the invention as specified in claim 72. 

123. With regard to claim 69, the combination of Kanbe et al and Sawa meets the limitations 
of claim 42 on which claim 69 depends. Furthermore, Kanbe et al discloses "relating a current 
situation state and current triggers to a course of action to determine the most appropriate update 
to the environment" in machine translated paragraph 55, where the manner in which the tile is 
thrown is depended on the feeling value and the state of the game. Kanbe et al discloses that the 
feeling value is incremented and decremented based on triggers as shown in machine-translated 
paragraph 42. The combination of Kanbe et al and Sawa does not show relating state and triggers 
to videos to determine the most appropriate update to the environment. Szeliski et al discloses 
"relating state and triggers to videos to determine the most appropriate update to the 
environment" {lines 4-9 of column 26: "As the user moves a slider (e.g., a time bar like on a 
video player) selecting a certain temporal portion of the video, the synthesis attempts to select 
frames that remain within that portion of the video, while at the same time using only fairly 
smooth transitions to jump back in time. "). 

124. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to relate videos to the current state and triggers as disclosed by Szeliski et al in addition to 
relating a course of action as disclosed by the Kanbe et al and Sawa et al combination. The 
motivation for doing so would have been to achieve a greater degree of realism by using video 
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animation instead of traditional animation, which Szeliski et al suggests is cheaper and easier 
than the alternative in lines 33-35 of column 29. Therefore, it would have been obvious to further 
modify the combination of Kanbe et al and Sawa et al with Szeliski et al to obtain the invention 
as specified in claim 69. 

125. Claims 48, 49, 50, 58, and 67 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kanbe et al in view of Sawa et al and in further view of Tateishi. 

126. With regard to claim 48, the combination of Kanbe et al and Sawa et al meets the 
limitations of claim 42, but does not teach controlling the character "in response to inaction of a 
human operator." Tateishi discloses "the operation of controlling the character in response to 
inaction of a human operator" (Fig, 12; lines 64-67 of column 12: "A display image 9B shows the 
state wherein the state of no command input relative to the dialogue options displayed in front of 
the character has been repeated by given integer times. "). 

127. With regard to claim 49, the combination of Kanbe et al and Sawa et al meets the 
limitations of claim 42, but does not teach "the operation of continuously controlling the 
character in response to actions and inaction of a human operator." Tateishi discloses "the 
operation of continuously controlling the character in response to actions and inaction of a 
human operator" (lines 36-39 of column 13: 'With this arrangement, the player can enjoy the 
character images and the background images which change in real time based on command 
inputs from the player. lines 10-13 of column 13: "As described above, when no command 
input is given within the idle time, different subsequent processes are carried out as compared 
with the case in which the command Input is given within the idle time. 
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128. With regard to claim 50, the combination of Kanbe et al and Sawa et al meets the 
limitations of claim 42, but does not teach controlling the character "in response to time between 
moves by the human operator." Tateishi discloses "the operation of controlling the character in 
response to the time between moves by the human operator" (Fig. 12; lines 64-67 of column 12: 
"A display image 9B shows the state wherein the state of no command input relative to the 
dialogue options displayed in front of the character has been repeated by given integer times. "). 

129. Kanbe et al, Sawa et al and Tateishi are analogous art because they are from the same 
problem solving area: simulating realistic interaction with computer animated objects. At the 
time of the invention it would have been obvious to a person of ordinary skill in the art to 
incorporate the aspect of time and user inaction, as disclosed by Tateishi, to the interaction 
between human and computer characters of the Kanbe et al and Sawa et al combination. The 
motivation for doing so would have been to make the computer-controlled characters seem more 
human to increase the realism of the interaction. Therefore, it would have been obvious to further 
modify the Kanbe et al and Sawa et al combination with Tateishi to obtain the invention as 
specified in claims 48, 49 and 50. 

130. With regard to claims 67 and 58, combination of Kanbe et al and Sawa et al meets the 
limitations of claim 66 on which claim 67 depends, and the limitations of claim 42 on which 
claim 58 depends. The combination of Kanbe et al and Sawa et al does not teach "the operation 
of putting a selection of the control of the character and a control of the background in place, and 
sending them to a display," as recited in claim 67, or making a "direct change to a character," as 
recited in claim 58. Tateishi discloses "the operation of putting a selection of the control of the 
character and a control of the background in place, and sending them to a display," as recited in 
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claim 67, which makes a "direct change to a character," as recited in claim 58 (as in c (SI 02 
shown in Fig. 6; lines 1-4 of column 10: "In case of the manual designation (Yes at step SI 01), 
the player selects and sets a character and background data via a given menu image (step 
S102)."). 

131. Kanbe et al, Sawa et al, and Tateishi are analogous art because they are from the same 
problem solving area: simulating realistic interaction with computer animated objects. At the 
time of the invention it would have been obvious to a person of ordinary skill in the art to add the 
operation of controlling the background and a character as disclosed by Tateishi to game 
disclosed by Kanbe et al and Sawa et al. The motivation for doing so would have been to allow 
the user to customize the interface, and provide variety to keep the user interested after several 
subsequent interactions. Therefore, it would have been obvious to further modify Kanbe et al and 
Sawa et al with Tateishi obtain the invention as specified in claims 67 and 58. 

132. Claim 60-63 are rejected under 35 U.S.C. 103(a) as being unpatentable over Kanbe 
et al in view of Sawa et al and in further view of Linnett et al. 

133. With regard to claim 60, as previously shown, the combination of Kanbe et al and Sawa 
et al show modifying a character based on triggers. The combination of Kanbe et al and Sawa et 
al does not specifically teach using a library of animations. Linnett et al discloses "the operation 
of modifying the character using a library of animations" (Fig. 11). Linnett et al discloses the 
operation of retrieving an animation 115 from a library 29. Linnett et al does not use the explicit 
language: modify a character; however, one of ordinary skill in the art would recognize that this 
feature is inherent from the statement on lines 52 -55 of column 7: "For personal character, a 
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separate animation is provided for each designated action that the character may perform or for 
each command to which the character is responsive." 

134. Kanbe et al, Sawa et al, and Linnett et al are analogous art because they are from the 
same problem solving area: simulating realistic interaction with computer animated objects. At 
the time of the invention it would have been obvious to a person of ordinary skill in the art to use 
the library of animations disclosed by Linnett et al to modify the characters in response to the 
actions of the player and other triggers disclosed by the Kanbe et al and Sawa et al combination. 
The motivation for doing so would have been to give the character a more personal feel thereby 
increasing the realism of the interaction; for instance, Linnett et al suggests that a character's 
animations can be can be used to create a personality for the character in lines 39-40 of column 
5. Therefore, it would have been obvious to further modify Kanbe et al and Sawa et al with 
Linnett et al obtain the invention as specified in claim 60. 

135. Claim 61 is met by the combination of Kanbe et al, Sawa et al and Linnett et al, wherein 
Linnett et al discloses "the character is an animated character, and wherein the animation is a 
cartoon" (Fig. 8 shows a cartoon dog; lines 41-43 of column 7: 'TIG. 8 shows a sequence of 
video frames 78, 80, 82 and 84 that are part of an animation for the dog personal character. "). 

136. With regard to claims 62 and 63, the Kanbe et al, Sawa et al and Linnett et al 
combination does not show that the animation is cell or clay animation; instead, the Kanbe et al, 
Sawa et al and Linnet et al combination shows the animation is cartoon. 

137. At the time the invention was made, it would have been an obvious matter of design 
choice to a person of ordinary skill in the art to use clay animation or cell animation because the 
applicant has not disclosed that using clay animation or cell animation provides an advantage, is 
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used for a particular purpose, or solves a stated problem. Furthermore, one of ordinary skill in 
the art would have expected the Kanbe et al, Sawa et al and Linnett et al combination to perform 
equally well with either cartoon animation as taught by Linnett et al or the claimed clay (claim 
63) or cell (claim 62) animation because all three types of animations would perform the same 
function of representing a character in an interaction equally well considering that they are all 
synthetic. 

138. Therefore, it would have been prima facie obvious to modify the Kanbe et al, Sawa et al 
and Linnett et al combination to obtain the invention as specified in claims 62 and 63 because 
such a modification would have been considered a mere design consideration which fails to 
patentably distinguish over the prior art of Kanbe et al, Sawa et al and Linnett et al. 

Conclusion 

139. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. U.S. Patent Application Publication 2002/01 18194 to Lanciault et al shows triggered 
non-linear animation. Arno Schodl, Richard Szeliski, David H. Salesin, Irfan Essa, "Video 
textures," July 2000, Proceedings of the 27th Annual Conference on Computer Graphics and 
Interactive Techniques, p.489-498 shows a "continuous infinitely varying stream of images" 
used for representing people, animals and other objects. Stephen Pollard, Sean Hayes, "3D Video 
Sprites," Feb. 1998, HPL-98-25, HP Labs Technical Report discloses streaming video with 
computer generative virtual environments. U.S. Patent No. 6,168,519 to Nakagawa et al shows a 
two-dimensional representation of a playing area and a three-dimensional representation of a 
playing area displayed simultaneously. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason M. Repko whose telephone number is 571-272-8624. The 
examiner can normally be reached on 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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